{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Deploying and Making Predictions with a Trained Model\n",
    "\n",
    "**Learning Objectives**\n",
    "- Deploy a model on Google CMLE\n",
    "- Make online and batch predictions with a deployed model\n",
    "\n",
    "## Introduction\n",
    "\n",
    "In this notebook, we will deploy the model we trained to predict birthweight and we will use that deployed model to make predictions using our cloud-hosted machine learning model. Cloud ML Engine provides two ways to get predictions from trained models; i.e., online prediction and batch prediction; and we do both in this notebook. \n",
    "\n",
    "Have a look at this blog post on [Online vs Batch Prediction](https://cloud.google.com/ml-engine/docs/tensorflow/online-vs-batch-prediction) to see the trade-offs of both approaches.\n",
    "\n",
    "As usual we start by setting our environment variables to reference our Project and Bucket. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "PROJECT = \"cloud-training-demos\"  # Replace with your PROJECT\n",
    "BUCKET = \"cloud-training-bucket\"  # Replace with your BUCKET\n",
    "REGION = \"us-central1\"            # Choose an available region for Cloud MLE\n",
    "TFVERSION = \"1.14\"                # TF version for CMLE to use"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ[\"BUCKET\"] = BUCKET\n",
    "os.environ[\"PROJECT\"] = PROJECT\n",
    "os.environ[\"REGION\"] = REGION\n",
    "os.environ[\"TFVERSION\"] = TFVERSION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "if ! gcloud storage ls --recursive gs://${BUCKET} | grep -q gs://${BUCKET}/babyweight/trained_model/; then\n",
    "    gcloud storage buckets create --location=${REGION} gs://${BUCKET}\n",
    "    # copy canonical model if you didn't do previous notebook\n",
    "    gcloud storage cp --recursive gs://cloud-training-demos/babyweight/trained_model gs://${BUCKET}/babyweight/trained_model\n",
    "fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy trained model\n",
    "\n",
    "Next we'll deploy the trained model to act as a REST web service using a simple gcloud call. To start, we'll check if our model and version already exists and if so, we'll delete them."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "MODEL_NAME=\"babyweight\"\n",
    "MODEL_VERSION=\"ml_on_gcp\"\n",
    "\n",
    "# Check to see if the model and version already exist, \n",
    "# if so, delete them to deploy anew\n",
    "if gcloud ai-platform models list | grep \"$MODEL_NAME \\+ $MODEL_VERSION\"; then\n",
    "    echo \"Deleting the version '$MODEL_VERSION' of model '$MODEL_NAME'\"\n",
    "    yes | gcloud ai-platform versions delete ${MODEL_VERSION} --model=$MODEL_NAME\n",
    "    \n",
    "    echo \"Deleting the model '$MODEL_NAME'\"\n",
    "    yes | gcloud ai-platform models delete ${MODEL_NAME}\n",
    "else \n",
    "    echo \"The model '$MODEL_NAME' with version '$MODEL_VERSION' does not exist.\"\n",
    "fi"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll now deploy our model. This will take a few minutes. Once the cell below completes, you should be able to see your newly deployed model in the 'Models' portion of the[ ML Engine section of the GCP console](https://pantheon.corp.google.com/mlengine/models)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "MODEL_NAME=\"babyweight\"\n",
    "MODEL_VERSION=\"ml_on_gcp\"\n",
    "MODEL_LOCATION=$(gcloud storage ls gs://${BUCKET}/babyweight/trained_model/export/exporter/ | tail -1)\n",
    "\n",
    "echo \"Deploying the model '$MODEL_NAME', version '$MODEL_VERSION' from $MODEL_LOCATION\"\n",
    "echo \"... this will take a few minutes\"\n",
    "\n",
    "gcloud ai-platform models create ${MODEL_NAME} --regions $REGION\n",
    "gcloud ai-platform versions create ${MODEL_VERSION} \\\n",
    "  --model ${MODEL_NAME} \\\n",
    "  --origin ${MODEL_LOCATION} \\\n",
    "  --runtime-version $TFVERSION"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use the deployed model to make online predictions\n",
    "\n",
    "To make online predictions, we'll send a JSON request to the endpoint of the service to make it predict a baby's weight. The order of the responses are the order of the instances."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from oauth2client.client import GoogleCredentials\n",
    "import requests\n",
    "import json\n",
    "\n",
    "MODEL_NAME = \"babyweight\"\n",
    "MODEL_VERSION = \"ml_on_gcp\"\n",
    "\n",
    "token = GoogleCredentials.get_application_default().get_access_token().access_token\n",
    "api = \"https://ml.googleapis.com/v1/projects/{}/models/{}/versions/{}:predict\" \\\n",
    "         .format(PROJECT, MODEL_NAME, MODEL_VERSION)\n",
    "headers = {\"Authorization\": \"Bearer \" + token }\n",
    "data = {\n",
    "  \"instances\": [\n",
    "    {\n",
    "      \"is_male\": \"True\",\n",
    "      \"mother_age\": 26.0,\n",
    "      \"plurality\": \"Single(1)\",\n",
    "      \"gestation_weeks\": 39\n",
    "    },\n",
    "    {\n",
    "      \"is_male\": \"False\",\n",
    "      \"mother_age\": 29.0,\n",
    "      \"plurality\": \"Single(1)\",\n",
    "      \"gestation_weeks\": 38\n",
    "    },\n",
    "    {\n",
    "      \"is_male\": \"True\",\n",
    "      \"mother_age\": 26.0,\n",
    "      \"plurality\": \"Triplets(3)\",\n",
    "      \"gestation_weeks\": 39\n",
    "    },\n",
    "    {\n",
    "      \"is_male\": \"Unknown\",\n",
    "      \"mother_age\": 29.0,\n",
    "      \"plurality\": \"Multiple(2+)\",\n",
    "      \"gestation_weeks\": 38\n",
    "    },\n",
    "  ]\n",
    "}\n",
    "response = requests.post(api, json=data, headers=headers)\n",
    "print(response.content)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When I ran the cell above, the predictions that I received for the four instances were **7.64**, **7.17**, **6.24** and **6.13** pounds, respectively. Your results might be different."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Use model for batch prediction\n",
    "\n",
    "Batch prediction is commonly used when you want to make thousands to millions of predictions at a time. To perform batch prediction we'll create a file with one instance per line and submit the entire prediction job through a `gcloud` command.\n",
    "\n",
    "To illustrate this, let's create a file `inputs.json` which has two instances on which we want to predict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile inputs.json\n",
    "{\"is_male\": \"True\", \"mother_age\": 26.0, \"plurality\": \"Single(1)\", \"gestation_weeks\": 39}\n",
    "{\"is_male\": \"False\", \"mother_age\": 26.0, \"plurality\": \"Single(1)\", \"gestation_weeks\": 39}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When making batch predictions, we specify the Google Cloud Storage location of the input json file as well as the locatin to deposit the predictions. The cell below submits a batch prediction job to the cloud. We can monitor the status from the 'Jobs' portion of the [ML Engine section of the GCP console](https://pantheon.corp.google.com/mlengine/jobs). Once the jobs shows that it's completed there, we can examine the predictions uploaded to the `OUTPUT` location we specify below."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%bash\n",
    "INPUT=gs://${BUCKET}/babyweight/batchpred/inputs.json\n",
    "OUTPUT=gs://${BUCKET}/babyweight/batchpred/outputs\n",
    "\n",
    "gcloud storage cp inputs.json $INPUT\n",
    "gcloud storage rm --recursive --continue-on-error $OUTPUT \n",
    "gcloud ai-platform jobs submit prediction babypred_$(date -u +%y%m%d_%H%M%S) \\\n",
    "    --data-format=TEXT \\\n",
    "    --region ${REGION} \\\n",
    "    --input-paths=$INPUT \\\n",
    "    --output-path=$OUTPUT \\\n",
    "    --model=babyweight \\\n",
    "    --version=ml_on_gcp"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Check the [AI Platform jobs submitted to the GCP console](https://pantheon.corp.google.com/mlengine/jobs) to make sure the prediction job has completed, then let's have a look at the results of our predictions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud storage ls gs://$BUCKET/babyweight/batchpred/outputs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!gcloud storage cat gs://$BUCKET/babyweight/batchpred/outputs/prediction.results*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Copyright 2017 Google Inc. Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
